Skip to content

fix(ci): treat a passed scoped run with no src coverage as legitimate, not a missing report#8195

Merged
JSONbored merged 1 commit into
mainfrom
fix/ci-scoped-coverage-guard
Jul 23, 2026
Merged

fix(ci): treat a passed scoped run with no src coverage as legitimate, not a missing report#8195
JSONbored merged 1 commit into
mainfrom
fix/ci-scoped-coverage-guard

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

What

The #8194 hardening: the third scoped-selection edge case in a week (the one that redded PR #8192's shards). The scoped run matched real tests that passed but exercised nothing under src/**, so --coverage.all=false wrote an empty lcov and 'Verify coverage report exists' failed a green run — no_tests_matched can't catch it because tests did match.

How

  • The scoped branch derives a second step output, no_src_coverage, from the run itself: the detector line is only reachable on vitest exit 0 (set -o pipefail), triggered by an absent/empty lcov, with the junit report as proof the suite ran. The success signal is the test run, never the coverage file's size.
  • All six verify/upload guards carry both escapes.
  • codecov-policy.test.ts pins the full condition strings and both detector lines, so neither escape can silently vanish — that's the regression-shape coverage ci: harden scoped test selection before re-enabling — third distinct red-CI edge case this week #8194 asks for.

Notes

Validation

actionlint green; codecov-policy suite green locally (8/8).

Advances #8194

…, not a missing report

The third scoped-selection edge case in a week (PR #8192's shards): the
selection matched real tests that PASSED but exercised nothing under
coverage.include (src/**) — a test-only/docs-drift diff — so
--coverage.all=false wrote an empty lcov and 'Verify coverage report
exists' failed a green run. no_tests_matched cannot see this case because
tests DID match.

The scoped branch now derives a second output (no_src_coverage) from the
run itself: reachable only on vitest exit 0, detected by the absent/empty
lcov, proven by the junit report the same command wrote. Every
verify/upload guard carries both escapes, and the codecov-policy drift
test pins the full condition strings plus both detector lines so neither
can silently vanish.

Scoped selection itself stays DISABLED via SCOPED_TEST_SELECTION_ENABLED=false
until the re-enable decision documented on the issue.

Advances #8194
@JSONbored JSONbored self-assigned this Jul 23, 2026
@JSONbored
JSONbored merged commit 5467233 into main Jul 23, 2026
5 checks passed
@JSONbored
JSONbored deleted the fix/ci-scoped-coverage-guard branch July 23, 2026 10:30
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
20726 4 20722 17
View the top 3 failed test(s) by shortest run time
test/unit/docs-selfhost-operations-runner-tmpdir.test.ts > self-hosting-operations doc: runner temp storage guidance matches docker-compose.yml (#selfhost-runner-tmp) > the doc's multi-runner snippet never mounts the Docker socket
Stack Traces | 0.00539s run time
AssertionError: expected null not to be null
 ❯ test/unit/docs-selfhost-operations-runner-tmpdir.test.ts:71:32
test/unit/docs-selfhost-operations-runner-tmpdir.test.ts > self-hosting-operations doc: runner temp storage guidance matches docker-compose.yml (#selfhost-runner-tmp) > includes a multi-runner override snippet whose x-runner-tmp-env values match the real anchor in docker-compose.yml
Stack Traces | 0.007s run time
AssertionError: expected a multi-runner CodeBlock snippet in the doc: expected null not to be null
 ❯ test/unit/docs-selfhost-operations-runner-tmpdir.test.ts:45:88
test/unit/docs-selfhost-audit-checklist.test.ts > self-host docs accuracy audit (#1829) > manifest lists every self-host docs route file on disk
Stack Traces | 0.0128s run time
AssertionError: docs.self-hosting-quickstart.tsx: expected false to be true // Object.is equality

- Expected
+ Received

- true
+ false

 ❯ test/unit/docs-selfhost-audit-checklist.test.ts:33:79
View the full list of 1 ❄️ flaky test(s)
test/unit/kill-switch-incident-runbook.test.ts > kill-switch incident runbook (#4809) > exists as a wired docs page with nav entry

Flake rate in main: 100.00% (Passed 0 times, Failed 9 times)

Stack Traces | 0.00657s run time
Error: ENOENT: no such file or directory, open '.../src/routes/docs.ams-kill-switch-incident.tsx'
 ❯ test/unit/kill-switch-incident-runbook.test.ts:19:19

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { errno: -2, code: 'ENOENT', syscall: 'open', path: '.../src/routes/docs.ams-kill-switch-incident.tsx' }

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: harden scoped test selection before re-enabling — third distinct red-CI edge case this week

1 participant